Designing Closed-Loop Models for Task Allocation

Automatically assigning tasks to people is challenging because human performance can vary across for many reasons. This challenge further compounded in real-life settings which no oracle exists assess the quality of decisions and task assignments made. Instead, we find ourselves a “closed” decision-making loop same fallible rely on practice must also be used guide allocation. How imperfect potentially biased train an accurate allocation model? Our key insight exploit weak prior information human-task similarity bootstrap model training. We show that use such improve accuracy, even when decision-makers are biased. present both theoretical analysis empirical evaluation over synthetic data social media toxicity detection task. Results demonstrate efficacy our approach.